Hyperspectral imaging is being introduced in a new medical imaging modality for the early and consistent detection of uterine cervical, colorectal, dermatological, esophageal and oral cancer. Hyperspectral refers to the instruments' ability to collect 20 to 30 times the color information as compared to a standard camera. This allows for discrimination between spectral features not normally available to a physician.
In hyperspectral imaging an entire scene is being imaged in a large number of spectral bands. The hyperspectral imaging device in this invention utilizes a technique called push-broom scanning. The hyperspectral sensor uses a progressive line scan to capture an entire image. For each scan line, the full spectrum for every pixel is provided. By taking a series of lines, a hyperspectral cube is developed. This hyperspectral cube contains spatial information in two dimensions (pixels) and spectral information in the third dimension.
Over the last decade, several groups have investigated the potential use of fluorescence and reflectance spectroscopy to detect neoplasia. Spectroscopic methods identify neoplasia by detecting biochemical and tissue architectural changes that are hallmarks of malignant transformation. Fluorescence spectroscopy detects endogenous fluorescent indicators of cellular metabolism, including NAD(P)H and FAD+, and pre-invasive changes to the connective tissue stroma, such as digestion of fluorescent collagen cross-links. Both reflectance and fluorescence spectroscopy can detect angiogenic changes, due to the spectral characteristics of hemoglobin.
Reflectance and fluorescence are two different modes of light interaction with matter. Reflectance is an elastic interaction process which means that there is no change in energy of the incident and the emitted light, while fluorescence is an inelastic interaction process which results in the emission of light with an energy different from that of the incident light. In reflectance mode, the tissue is illuminated using a broadband white light source in the visible light spectrum, and detects the reflected intensity in the same spectral region. In fluorescence mode, the tissue is excited with narrowband Ultraviolet (UV) light and collects fluorescence in the visible spectral region.
The calculation of intrinsic fluorescence, the “pure” fluorescence, requires the spatial registration of the reflectance and fluorescence hyperspectral imagery.
Advanced computer algorithms to diagnose pre-cancerous and cancerous tissue regions utilize the fusion of multiple data sources, including the reflectance and fluorescence hyperspectral imagery, to optimize their performance. A key enabling technology for the data fusion is the registration of the different data sources.
Using a push-broom scanning technology, the hyperspectral bands are already aligned to each other and only the two spatial dimensions of the hyperspectral imagery needs to be registered. The two spatial dimensions can be represented by a 2D image from a single band or a 2D image calculated from any number of bands.
The image registration of two images involves the matching of features present in both images, and from their spatial relationships the calculation of the image transformation between them.
The fluorescence and reflectance hyperspectral imagery by their nature exhibit different features. The ambiguity in multi-modal image registration, due to the different features present in the images, typically only allows to do image registration using an affine image transformation (rotation, translation and scale). Unfortunately, between the fluorescence and reflectance hyperspectral data acquisition the patient and in particular the body part under examination may have moved. This is soft tissue movement that cannot be described with an affine image transformation and requires a general (elastic) warp image transformation.
The amplitude of this movement and therefore the need for registration can be minimized or even eliminated by minimizing the time between the acquisition of the fluorescence and reflectance hyperspectral imagery for a given pixel (spatial location). This can be achieved with a system design collecting the fluorescence and reflectance hyperspectral imagery in a scan-line interleaved manner. For each scan-line the fluorescence and reflectance hyperspectral data is acquired before moving to the next scan-line.
A different system design, considered in this invention, collects the fluorescence and reflectance hyperspectral imagery in a sequential manner; one complete hyperspectral data set after the other, allowing for more ample tissue movement and requiring robust image registration.
The invention provides systems and methods to embed a reflectance image in the fluorescence hyperspectral data, allowing the image registration of fluorescence and reflectance hyperspectral imagery to use a general (elastic) warp image transformation. By embedding a reflectance image into the fluorescence hyperspectral imagery, both data sets now have a resembling reflectance image. This resolves the ambiguity of the image registration and the data sets can be registered using a general (elastic) warp image transformation taken into account the soft tissue movement.
The invention is not limited to any particular image registration algorithm but rather foresees the use of any image registration algorithm that is appropriate for the targeted application. General surveys [Brown 1992], [Maurer 1993], [van den Elsen 1993], [West 97], [Maintz 1998], [Lester 1999], [Rohr 2000] and [Hill 2001] provide an extensive list of suitable algorithms that can be used.